Dynamically adjusting statistics collection time in a database management system

ABSTRACT

Each of one or more commit cycles may be associated with a predicted number of updates. A statistics collection time for a database table can be determined by estimating a sum of predicted updates included in one or more commit cycles. Whether the estimated sum of predicted updates is greater than a first threshold may be determined. In addition, a progress point for a first one of the commit cycles can be determined. A time to collect statistics may be selected based on the progress point of the first commit cycle.

BACKGROUND

This disclosure relates generally to database management systems, andmore specifically, to adjusting the time that a statistics collectioncomponent of a database management system updates database statistics.

A relational database organizes data in two dimensions, i.e., as tableshaving rows and columns. A relational database is typically used in anenterprise to store large quantities of transactional data. Userprograms can be used to access a database. An access may read or writedata to the database. In addition, an access may include searching forparticular data. Structured Query Language (SQL) is a language that maybe used to access data stored in a relational database.

Many database management systems (DBMS) include a query optimizer and astatistics collection component to collect various statistics for datatables. Some example statistics include the number of rows in a table,the number of distinct values in a column (cardinality), the mostfrequent values in a column, and the distribution of data values. Thequery optimizer uses table statistics to generate and select an accessplan for executing a particular query. The query optimizer may select a“best” plan from several alternative plans, e.g., a plan that isfastest, uses fewest resources, and the like.

SUMMARY

Various embodiments are directed to a computer-implemented method fordetermining a statistics collection time for a database table. Themethod may include estimating a sum of predicted updates included in oneor more commit cycles. Each of the commit cycles is associated with apredicted number of updates. In addition, it may be determined whetherthe estimated sum of predicted updates is greater than a firstthreshold. The method may also include determining a progress point fora first one of the commit cycles and selecting a time to collectstatistics based on the progress point of the first commit cycle.

In addition, various embodiments are directed to a system and a computerprogram product for determining a statistics collection time for adatabase table.

The above summary is not intended to describe each illustratedembodiment or every implementation of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a computing device in which variousembodiments may be implemented.

FIG. 2 is a conceptual illustration of the major software components ofthe computing device of FIG. 1 according to various embodiments.

FIG. 3 is a flowchart of a process for monitoring a commit cycle duringits life time and maintaining a commit cycle queue according to variousembodiments.

FIGS. 4A and 4B depict a flowchart of process 400 for dynamicallydetermining a statistics collection time for a database table accordingto various embodiments.

In the Figures and the Detailed Description, like numbers refer to likeelements.

The drawings included in the present application are incorporated into,and form part of, the specification. They illustrate embodiments of thepresent disclosure and, along with the description, serve to explain theprinciples of the disclosure. The drawings are only illustrative ofcertain embodiments and do not limit the disclosure. While the inventionis amenable to various modifications and alternative forms, specificsthereof have been shown by way of example in the drawings and will bedescribed in detail. It should be understood, however, that theintention is not to limit the invention to the particular embodimentsdescribed. On the contrary, the intention is to cover all modifications,equivalents, and alternatives falling within the spirit and scope of theinvention.

DETAILED DESCRIPTION

Aspects of the present disclosure relate to database management systems,and more specifically, to adjusting the time that a statisticscollection component of a DBMS updates database statistics. While thepresent disclosure is not necessarily limited to such applications,various aspects of the disclosure may be appreciated through adiscussion of various examples using this context.

In a DBMS, a query optimizer uses statistics collected by a statisticscollection component to generate and select an access plan for executinga query. Generally, it is important that the statistics provided to thequery optimizer be accurate. If an access plan is selected based onstale or outdated statistics, then some access plan other than the“best” access plan may be selected. In comparison with the best plan, anaccess plan that is not the best may require longer processing time oruse more resources. Statistics may become inaccurate or stale as aresult of changes that are made to the underlying data after thestatistics are collected.

When a statistics collection component in a DBMS operates, it consumessystem resources. Collecting statistics can require scanning and sortingall of the data in a table. Collecting statistics can be resourceintensive, especially for large tables. When a statistics collectioncomponent is operating, user programs and batch jobs may need to shareaccess to a database table with the component.

User programs can be used to read or write data to a database. Userprograms often perform reads and writes in a series of steps that may bereferred to as a transaction. A transaction is a logical unit of workthat may include a “commit cycle.” During the first phase of a commitcycle, a user program modifies data stored in a database table. Thesemodifications may be tracked using database logs and can be “rolledback” so long as the commit cycle has not completed. In other words, themodifications have only a “temporary” character during the first phaseof a commit cycle. If the transaction fails at some point during thefirst phase of the commit cycle, all of the changes made during thefirst phase need to be undone. Rolling back is a process that backs outchanges made by a failed transaction. In the second phase, themodifications to database are “committed” or rolled back. In the secondphase, the modified data becomes “permanent” once it has been committed.Once modifications are committed, they can no longer be rolled back.

A user program may include one or several large transactions withthousands or even millions of database updates. This is especially truefor some batch jobs. At the end of the first phase of the commit cyclefor a user program, the program or batch job may issue a commit command.However, if something goes wrong during the first phase, perhaps nearthe end of the first phase (after thousands or millions of rows havebeen modified), the program or batch job issues a roll back command.

A statistics collection component may be configured to determine whenstatics are stale and to update statistics when it determines thatstatics have become stale. The statistics collection component maymonitor the number of modifications recorded in a database log todetermine when statics are stale. A statistics collection component maydetermine that statics are stale during a first phase of a commit cycleinvolving a large number of updates. As a result, the statisticscollection component may collect updated statistics during the firstphase of the large job. There may be problems, however, if the updatesare rolled back in the second phase of the commit cycle. The roll backmakes the updated statistics collected in phase one inaccurate. Inaddition, the pre-update statistics remain stale and the statisticscollection component needs to repeat the process of collectingstatistics. This wastes system resources. In addition, query plans mayperform poorly until the statistics can be refreshed.

According to various embodiments, a statistics collection componentdetermines a time for collecting statistics for a database table. Insome embodiments, as further described below, a commit cycle may bepredicted to have a large number of updates. If it is determined that aparticular commit cycle, e.g., one that is predicted to have a largenumber of updates, has not yet started, a statistics collection time canbe set at an immediate time. In various embodiments, statisticscollection can be set to start immediately, even if the typicalstatistics collection time might have been scheduled for a later time.If a commit cycle predicted to have a large number of updates ispredicted to complete soon, a statistics collection time can be set to afuture time when the commit cycle has completed. If a commit cyclepredicted to have a large number of updates is determined to be in the“middle” of its life cycle, a statistics collection time can be set to afuture time or statistics can be collected at an immediate time usingsampling if there is a relatively “urgent” need for updated statistics.

In various embodiments, a method for determining a time for collectingstatistics can include estimating a sum of predicted updates included inone or more commit cycles. In addition, the method may includedetermining whether the estimated sum of predicted updates is greaterthan a first threshold, determining a progress point for a first one ofthe commit cycles, and selecting a time to collect statistics based onthe progress point of the first commit cycle.

FIG. 1 is a block diagram of a computing device 100 according to variousembodiments. The computing device 100 is one example of a context inwhich embodiments may be implemented. The mechanisms and apparatus ofthe various embodiments disclosed herein apply equally to anyappropriate computing device. The major components of the computingdevice 100 include one or more processors 102, a memory 104, a terminalinterface 112, a storage interface 114, an Input/Output (“I/O”) deviceinterface 116, and a network interface 118, all of which arecommunicatively coupled, directly or indirectly, for inter-componentcommunication via a memory bus 106, an I/O bus 108, bus interface unit(“IF”) 109, and an I/O bus interface unit 110.

The computing device 100 may contain one or more general-purposeprogrammable central processing units (CPUs) 102A and 102B, hereingenerically referred to as the processor 102. In an embodiment, thecomputing device 100 may contain multiple processors; however, inanother embodiment, the computing device 100 may alternatively be asingle CPU device. Each processor 102 executes instructions stored inthe memory 104 and may include one or more levels of on-board cache.

In an embodiment, the memory 104 may include a random-accesssemiconductor memory, storage device, or storage medium (either volatileor non-volatile) for storing or encoding data and programs. In anotherembodiment, the memory 104 represents the entire virtual memory of thecomputing device 100, and may also include the virtual memory of othercomputer systems coupled to the computing device 100 or connected via anetwork 124. The memory 104 is conceptually a single monolithic entity,but in other embodiments the memory 104 is a more complex arrangement,such as a hierarchy of caches and other memory devices. For example,memory 104 may exist in multiple levels of caches, and these caches maybe further divided by function, so that one cache holds instructionswhile another holds non-instruction data, which is used by the processoror processors. Memory 104 may be further distributed and associated withdifferent CPUs or sets of CPUs, as is known in any of various so-callednon-uniform memory access (NUMA) computer architectures.

The memory 104 may store all or a portion of the software components anddata shown in FIG. 2. These programs and data structures are illustratedin FIG. 2 as being included within the memory 104 in the computingdevice 100, however, in other embodiments, some or all of them may be ondifferent computer systems and may be accessed remotely, e.g., via anetwork 124. The computing device 100 may use virtual addressingmechanisms that allow the programs of the computing device 100 to behaveas if they only have access to a large, single storage entity instead ofaccess to multiple, smaller storage entities. Thus, while the componentsand data shown in FIG. 2 are illustrated as being included within thememory 104, these components and data are not necessarily all completelycontained in the same storage device at the same time. Further, althoughthe components and data shown in FIG. 2 are illustrated as beingseparate entities, in other embodiments some of them, portions of someof them, or all of them may be packaged together.

In an embodiment, the components and data shown in FIG. 2 may includeinstructions or statements that execute on the processor 102 orinstructions or statements that are interpreted by instructions orstatements that execute on the processor 102 to carry out the functionsas further described below. In another embodiment, the components shownin FIG. 2 may be implemented in hardware via semiconductor devices,chips, logical gates, circuits, circuit cards, and/or other physicalhardware devices in lieu of, or in addition to, a processor-basedsystem. In an embodiment, the components shown in FIG. 1 may includedata in addition to instructions or statements.

The computing device 100 may include a bus interface unit 109 to handlecommunications among the processor 102, the memory 104, a display system124, and the I/O bus interface unit 110. The I/O bus interface unit 110may be coupled with the I/O bus 108 for transferring data to and fromthe various I/O units. The I/O bus interface unit 110 communicates withmultiple I/O interface units 112, 114, 116, and 118, which are alsoknown as I/O processors (IOPs) or I/O adapters (IOAs), through the I/Obus 108. The display system 124 may include a display controller, adisplay memory, or both. The display controller may provide video,audio, or both types of data to a display device 126. The display memorymay be a dedicated memory for buffering video data. The display system124 may be coupled with a display device 126, such as a standalonedisplay screen, computer monitor, television, or a tablet or handhelddevice display. In an embodiment, the display device 126 may include oneor more speakers for rendering audio. Alternatively, one or morespeakers for rendering audio may be coupled with an I/O interface unit.In alternate embodiments, one or more of the functions provided by thedisplay system 124 may be on board an integrated circuit that alsoincludes the processor 102. In addition, one or more of the functionsprovided by the bus interface unit 109 may be on board an integratedcircuit that also includes the processor 102.

The I/O interface units support communication with a variety of storageand I/O devices. For example, the terminal interface unit 112 supportsthe attachment of one or more user I/O devices 120, which may includeuser output devices (such as a video display device, speaker, and/ortelevision set) and user input devices (such as a keyboard, mouse,keypad, touchpad, trackball, buttons, light pen, or other pointingdevice). A user may manipulate the user input devices using a userinterface, in order to provide input data and commands to the user I/Odevice 120 and the computing device 100, and may receive output data viathe user output devices. For example, a user interface may be presentedvia the user I/O device 120, such as displayed on a display device,played via a speaker, or printed via a printer.

The storage interface 114 supports the attachment of one or more diskdrives or direct access storage devices 122 (which are typicallyrotating magnetic disk drive storage devices, although they couldalternatively be other storage devices, including arrays of disk drivesconfigured to appear as a single large storage device to a hostcomputer, or solid-state drives, such as flash memory). In anotherembodiment, the storage device 122 may be implemented via any type ofsecondary storage device. The contents of the memory 104, or any portionthereof, may be stored to and retrieved from the storage device 122 asneeded. The I/O device interface 116 provides an interface to any ofvarious other I/O devices or devices of other types, such as printers orfax machines. The network interface 118 provides one or morecommunication paths from the computing device 100 to other digitaldevices and computer systems; these communication paths may include,e.g., one or more networks 124.

Although the computing device 100 shown in FIG. 1 illustrates aparticular bus structure providing a direct communication path among theprocessors 102, the memory 104, the bus interface 109, the displaysystem 124, and the I/O bus interface unit 110, in alternativeembodiments the computing device 100 may include different buses orcommunication paths, which may be arranged in any of various forms, suchas point-to-point links in hierarchical, star or web configurations,multiple hierarchical buses, parallel and redundant paths, or any otherappropriate type of configuration. Furthermore, while the I/O businterface unit 110 and the I/O bus 108 are shown as single respectiveunits, the computing device 100 may, in fact, contain multiple I/O businterface units 110 and/or multiple I/O buses 108. While multiple I/Ointerface units are shown, which separate the I/O bus 108 from variouscommunications paths running to the various I/O devices, in otherembodiments, some or all of the I/O devices are connected directly toone or more system I/O buses.

In various embodiments, the computing device 100 is a multi-usermainframe computer system, a single-user system, or a server computer orsimilar device that has little or no direct user interface, but receivesrequests from other computer systems (clients). In other embodiments,the computing device 100 may be implemented as a desktop computer,portable computer, laptop or notebook computer, tablet computer, pocketcomputer, telephone, smart phone, or any other suitable type ofelectronic device.

FIGS. 1 and 2 are intended to depict the representative major componentsof the computing device 100. Individual components, however, may havegreater complexity than represented in FIG. 1, components other than orin addition to those shown in FIG. 1 may be present, and the number,type, and configuration of such components may vary. Several particularexamples of additional complexity or additional variations are disclosedherein; these are by way of example only and are not necessarily theonly such variations. The various program components illustrated in FIG.2 may be implemented, in various embodiments, in a number of differentmanners, including using various computer applications, routines,components, programs, objects, modules, data structures, etc., which maybe referred to herein as “software,” “computer programs,” or simply“programs.”

FIG. 2 is a conceptual illustration of the major software components ofcomputing device 100 in memory 104. Operating system 201 providesvarious low-level software functions, such as device interfaces,management of memory pages, management and dispatching of multipletasks, etc. as is well-known in the art. A structured database 202contains data which is maintained by computing device 100 and for whichthe computing device provides access to one or more users, who may bedirectly attached to computing device 100 or may be remote clients whoaccess computing device 100 through a network using a client/serveraccess protocol. In some embodiments, database 202 is a relationaldatabase. In other embodiments, database 202 may be any other known typeof database, such as key-value data store. Database 202 contains one ormore tables, each having a plurality of records, each record containingat least one (and usually many) fields, as is well known in the art.Database 202 might contain almost any type of data which is provided tousers by a computer system. Associated with database 202 are multipledatabase indexes 203-205, each index representing an ordering of recordsin database 202 according to some specified criterion. Although only onedatabase 202 and three indexes 203-205 are shown in FIG. 2, thecomputing device may contain multiple databases, and the number ofindexes may vary (and typically is much larger). Alternatively, database202 on computing device 100 may be logically part of a largerdistributed database which is stored on multiple computer systems.

Database management system 211 provides basic functions for themanagement of database 202. Database management system 211 maytheoretically support an arbitrary number of databases, although onlyone is shown in FIG. 2. Database management system 211 preferably allowsusers to perform basic database operations, such as defining a database,altering the definition of the database, creating, editing and removingrecords in the database, viewing records in the database, definingdatabase indexes, and so forth. Among the functions supported bydatabase management system 211 is the making of queries against data indatabase 202. Query support functions in database management system 211include query optimizer 212, query execution engine 213, and statisticscollection component (statistics collector) 215. The statisticscollection component may include logic and a commit cycle (“CC”) queue217, further described below. In various embodiments, the CC queue neednot be a queue, but may be any suitable type of data structure. Databasemanagement system 211 may further contain any of various more advanceddatabase functions. Although database management system 211 isrepresented in FIG. 2 as an entity separate from operating system 201,it will be understood that in some computer architectures variousdatabase management functions are integrated with the operating system.

Query optimizer 212 generates query execution strategies for performingdatabase queries. The amount of time or resource required to perform acomplex query on a large database can vary greatly, depending on variousfactors, such as the availability of an index, the amount of resourcesrequired to evaluate each condition, and the expected selectivity (i.e.,number of records eliminated from consideration) of the various logicalconditions. Query optimizer 212 determines an “optimal” executionstrategy according to any optimizing algorithm, now known or hereafterdeveloped, and generates an execution strategy, also known as an “accessplan” or “query plan,” according to the determination. The queryoptimizer 212 receives various statistics for data tables from thestatistics collection component 215. Some example statistics include thenumber of rows in a table, the number of distinct values in a column(cardinality), the most frequent values in a column, and thedistribution of data values. The query optimizer 212 uses tablestatistics to generate and select an access plan for executing aparticular query.

A query can be saved as a persistent storage object in memory, and canbe written to disk or other storage. Once created by query optimizer212, a query execution strategy can be saved with the query as part ofthe persistent storage object. For a given query, it is possible togenerate and save one or multiple optimized execution strategies. Thequery can be invoked, and a saved query strategy re-used (re-executed),many times.

FIG. 2 represents persistent storage objects Query A 206 and Query B207. Although two query objects, each containing two executionstrategies, are represented for illustrative purposes in FIG. 2, it willbe understood that the actual number of such entities may vary, thattypically a large computer system contains a much larger number of queryobjects, that each query object may contain or be associated with zero,one, two, or more than two execution strategies. Although these arereferred to herein as “query objects”, the use of the term “object” isnot meant to imply that database management system 211 or othercomponents are necessarily programmed using so-called object-orientedprogramming techniques, or that the query object necessarily has theattributes of an object in an object-oriented programming environment,although it would be possible to implement them using object-orientedprogramming constructs.

In addition to database management system 211, one or more userapplications 214, 215 and one or more batch jobs 208, 209 executing onCPU 101 may access data in database 202 to perform tasks on behalf ofone or more users. Such user applications may include, e.g., personnelrecords, accounting, code development and compilation, mail,calendaring, or any of thousands of user applications. Some of theseapplications may access database data in a read- only manner, whileothers have the ability to update data. There may be many differenttypes of read or write database access tasks, each accessing differentdata or requesting different operations on the data. For example, onetask may access data from a specific, known record, and optionallyupdate it, while another task may invoke a query, in which all recordsin the database are matched to some specified search criteria, data fromthe matched records being returned, and optionally updated. Furthermore,data may be read from or written to database 202 directly, or mayrequire manipulation or combination with other data supplied by a user,obtained from another database, or some other source. In yet anotherexample, hundreds of thousands or millions of pending financialtransactions may be collected in one or more transaction filesthroughout the day. At the end of the business day, the pendingtransactions may be posted to customer accounts maintained in a databaseas a large batch job. A batch job may read and write data to database202. Although two applications 214, 215 are shown for illustrativepurposes in FIG. 2, the number of such applications may vary.Applications 214, 215 typically utilize function calls to databasemanager 211 to access data in database 202, and in particular, toexecute queries to data in the database, although in some systems it maybe possible to independently access data in database 202 directly fromthe application.

Various software entities are represented in FIG. 2 as being separateentities or contained within other entities. However, it will beunderstood that this representation is for illustrative purposes only,and that particular modules or data entities could be separate entities,or part of a common module or package of modules. Furthermore, althougha certain number and type of software entities are shown in theconceptual representation of FIG. 2, it will be understood that theactual number of such entities may vary, and in particular, that in acomplex database server environment, the number and complexity of suchentities is typically much larger. Additionally, although softwarecomponents are depicted in FIG. 2 as residing on the single computingdevice 100 for completeness of the representation, it is not necessarilytrue that all programs, functions and data will be present on a singlecomputer computing device or will be performed on a single computercomputing device. For example, user applications may be on a separatesystem from the database; a database may be distributed among multiplecomputer systems, so that queries against the database are transmittedto remote systems for resolution, and so forth.

While the software components of FIG. 2 are shown conceptually asresiding in memory 104, it will be understood that in general the memoryof a computer system will be too small to hold all programs and datasimultaneously, and that information is typically stored in one or moredata storage devices 122, comprising one or more mass storage devicessuch as rotating magnetic disk drives, and that the information is pagedinto memory by the operating system as required. In particular, database202 is typically much too large to be loaded into memory, and typicallyonly a small portion of the total number of database records is loadedinto memory at any one time. The full database 202 is typically recordedin disk storage. Furthermore, it will be understood that the conceptualrepresentation of FIG. 2 is not meant to imply any particular memoryorganizational model, and that computing device 100 might employ asingle address space virtual memory, or might employ multiple virtualaddress spaces which overlap.

FIG. 3 is a flowchart of a process 300 for monitoring a commit cycleduring its life time and maintaining a commit cycle queue 217 accordingto various embodiments. As a preliminary matter, it should be noted thateach commit cycle may have a unique identifier and a particular commitcycle may be executed once or multiple times. In addition, the commitcycle queue 217 created and updated in process 300 is used by a process400 described below in FIGS. 4A, 4B. In operation 302, a commit cyclefor a particular database table is received. It is determined inoperation 304 whether a signature for the received commit cycle isknown, i.e., whether an existing signature is associated with the commitcycle. A “signature” for a commit cycle may include the number ofdatabase updates processed during a previous execution of the commitcycle, as well as other information, such as the time or resources usedby the commit cycle and whether the commit cycle was rolled back. Thesignature may be based on the program or function name and/or offsetstherein, and/or a portion of the program stack, or particular start/endoffsets of a transaction in a particular program. In general, thesignature uniquely identifies the CC of interest. The signature may beused as a key or offset to metadata for the CC. In the CC metadata,there will be more particular data such as the size (number of updates),average size (average number of updates), or weighted average size(weighted average number of updates), high/low size, number of resourcesused, and other similar metrics. Metadata may also include highest,lowest, and other similar sizes of past executions of the CC. It isthrough its associated signature that a commit cycle is associated witha predicted number of updates or, more generally, a predicted amount ofresource usage. In cases where other information is included in thesignature, the signature for a commit cycle may combine the otherinformation from previous executions. For example, the signature for acommit cycle may include an average number of database updates or thepercentage of times that the commit cycle was rolled back in previousexecutions. To simplify the following discussion, it may be assumed thatthe signature only includes the number of updates or the average numberof updates. However, it should be understood that in various embodimentsthe signature may be a score that combines a number or average number ofupdates with other information, such as processing time or resourcesused, and/or the likelihood or probability of a roll back. Theprobability of a roll back may be based on the number of times previousexecutions of the commit cycle have been rolled back or the number oftimes previous executions of the commit cycle have been committed.

If a signature is determined to be known, it is obtained in operation306 and the process 300 proceeds to operation 308. For example, asignature may be read from a data structure in a memory. If a signatureis determined to be unknown, the process 300 proceeds directly tooperation 308. A signature may be unknown, for example, when it is thefirst time the particular commit cycle is executed. When the signaturefor a commit cycle is unknown, a default signature may be associatedwith the commit cycle. When a default signature is associated with acommit cycle, a predicted number of updates is implicitly associatedwith the commit cycle. In operation 308, the commit cycle is insertedinto commit cycle queue 217. If the signature is known, the commit cycleis placed in the queue based on its size (or score). (As used herein,the “size” of a CC is the number of database changes associated with theCC.) For example, the commit cycle queue may arrange active commitcycles by size. If the current commit cycle is larger than any othercommit cycle, it is placed in the commit queue 217 at the beginning (orend) of the list. In cases where the signature of the commit cycle isnot known, it may be placed in a position in commit cycle queue 217based on a policy and/or its default signature. For example, a commitcycle with a default signature may be placed at the beginning, end, ormiddle of the queue based on a predicted number of updates in thedefault signature.

In operation 310, one or more commit cycles are processed in parallelaccording to known methods. As commit cycles are processed, the numberof uncommitted updates are tracked or monitored in operation 312.Uncommitted updates are updates occurring in the first phase of a commitcycle. The tracking or monitoring of updates may include determiningwhether the number of updates diverges from the number of updatesestimated or predicted by the signature for the commit cycle. Thetracking or monitoring of updates may also include determining whetherthe commit cycle has not yet started and whether the commit cycle hascompleted. In operation 316, it is determined whether the size, i.e.,number of updates, of the commit cycle is greater or smaller than theestimated number of updates. If the number of updates diverges from thenumber estimated or predicted by the signature, the process moves tooperation 318 in which an updated estimate of the size of the commitcycle is made. For example, the signature for a particular commit cyclemay predict 1,000 updates. Monitoring, however, detects 2,000 pendingupdates. The estimate of the size of the commit cycle may be updated tothe current number of pending updates, i.e., 2,000. If the number ofpending updates for the particular commit cycle continues to increase,the estimated size of the commit cycle may be updated multiple times. Invarious embodiments, operation 316 may determine whether the size of thecommit cycle is larger or smaller than estimated or predicted.

In operation 320, the commit cycle queue 217 is re-sorted. As mentioned,the commit cycle queue 217 is used by a process 400 described below inFIGS. 4A, 4B. Active commit cycles and commit cycles scheduled to beactive are arranged by size in the commit cycle queue 217 so that theprocess 400 may operate on large commit cycles before small commitcycles. A commit cycle that is active is one that is currentlyexecuting. A commit cycle scheduled to be active is one that is justabout to start executing. In other words, it is a commit cycle that willbegin executing in the near future. For example, assume the commit cyclequeue 217 holds 100 commit cycles. In addition, assume that thesignature for one of these commit cycles includes a prediction of onemillion updates. Further assume that the signatures of the remaining 99commit cycles predict that each holds 10 updates. Sorting the commitcycle queue 217 causes the process 400 to address the large commit cyclehaving one million updates before the remaining 99 commit cycles.Continuing the example, assume that during the processing of the 99commit cycles predicted to hold 10 updates, monitoring in operation 316determines that the size of one of the commit cycle diverges from theestimate. Assume that operation 316 determines that this commit cyclehas 10,000 pending updates, instead of the estimated 10. Resorting thecommit cycle queue in operation 320 allows the process 400 to operate onthis large commit cycle before commit cycles predicted to have a smallernumber of updates.

In operation 314, it is determined whether the commit cycle has ended.In other words, it is determined whether the commit cycle has eithercommitted or rolled back its pending updates. In operation 322, thecommit cycle is removed from the queue. In operation 324, a signaturemay be determined and either saved or updated for the commit cycleinstance. A signature may be determined or updated based on an actualnumber of updates or resources used during execution.

FIGS. 4A and 4B depict a flowchart of process 400 for dynamicallydetermining a statistics collection time for a database table accordingto various embodiments. In operation 402, a determination is made as towhether a set of statistics for a table are stale. Statistics may bedetermined to be stale using any method now known or hereafter devised.For example, committed or pending updates, or both committed and pendingupdates may be tracked. When the number of updates tracked is greaterthan a threshold number of updates, statistics for the table may bedetermined to be stale. As another example, when a particular amount oftime has elapsed, the statistics for the table may be determined to bestale. When it is determined that a set of statistics are stale, i.e.,not current, the process 400 advances to operation 404.

In a first iteration of operations 404-412, a signature of a largestpending commit cycle is fetched or read from the commit cycle queue 217in operation 404. In operation 406, it is determined whether this commitcycle is estimated to include a number of updates that is larger than athreshold number of updates (a second size threshold “S2”). In analternative embodiment, a signature may include a score (as describedabove) and it is determined in operation 406 whether the score for thecommit cycle is larger than an S2 threshold score. If the estimatednumber or updates (or score) is larger than the second size threshold,the process may move to optional operation 408 or operation 410 ifoperation 408 is omitted.

In optional operation 408, it may be determined whether a prediction orestimate of likelihood of a roll back of the commit cycle is greaterthan a confidence threshold. For example, if a particular commit cyclehas been rolled back 50 percent of the time in its previous executionsand the confidence threshold is 20 percent, it may be determined inoperation 408 that a roll back is somewhat likely and process 400advances operation 410. As a second example, if the particular commitcycle has been rolled back 10 percent of the time in its previousexecutions and the confidence threshold is 15 percent, it may bedetermined in operation 408 that a roll back is not somewhat likely andprocess 400 returns to operation 404. Optional operation 408 may beincluded in process 400 to exclude commit cycles that are unlikely to berolled back from a determination of whether a statistics collection timeshould be scheduled immediately or later.

A sum of predicted updates in the commit cycles currently in the commitcycle queue 217 is estimated and updated in operation 410. For example,if a largest commit cycle is estimated to include 100,000 updates, asecond largest commit cycle is estimated to include 50,000 updates, anda third largest commit cycle is estimated to include 25,000 updates,then, after three iterations of operations 404-412, the sum of predictedupdates would be 175,000 updates after three iterations of operations404-412 (ignoring optional operation 408).

In operation 412, it is determined whether the commit cycle in thecurrent iteration is the last commit cycle in the commit cycle queue217. If there are additional commit cycles in the commit cycle queue217, the process 400 returns to operation 404 where the next largestcommit cycle is fetched. Otherwise, the process 400 advances tooperation 416 if all commit cycles in the commit cycle queue 217 havebeen processed.

Operation 414 is reached when the operation 406 determines that thecommit cycle in the current iteration includes a predicted number ofupdates that is less than or equal to the S2 threshold number of updatesor score (second size threshold). Recall the example in which the commitcycle queue 217 holds 100 commit cycles and the largest commit cycle isestimated to have one million updates while the remaining 99 commitcycles are estimated to each hold 10 updates. Assume the second sizethreshold is 100 updates. In a first iteration of operations 404-412,the largest commit cycle would be processed and used to update the sumof predicted updates. In a second iteration, a first one of theremaining 99 commit cycles would be determined to be below the secondsize threshold. In operation 414, in one embodiment, the number ofcommit cycles in the commit queue 217 is determined and this number maybe multiplied by an average small commit cycle size. Continuing theexample, if the average small commit cycle size is 10, an estimate ofthe number of predicted updates for commit cycles below the second sizethreshold is determined to be 990. The estimate of 990 predicted updateswould be added to the sum of predicted updates in operation 414.

In operation 416, it is determined whether the sum of predicted updatesis larger than a first size threshold (“S1”). If the sum of pendingupdates is less than or equal to the first size threshold, collection ofstatistics need not be delayed and may begin immediately, as indicatedby operation 418. On the other hand, if the sum of predicted updates isgreater than the first size threshold, the statistics are in need ofupdating and subsequent operations may be employed to determine a timefor updating the set of statistics. Operation 416 may refine operation402, serve as a double check on operation 402, or be employed in lieu ofoperation 402.

In various embodiments, a progress point for a commit cycle isdetermined. The operations 420, 422, and 424 described below mayemployed to determine a progress point for a first one of the commitcycles, according to an embodiment. The operations 420, 422, and 424 maybe performed for each commit cycle in the commit cycle queue 217 in someembodiments. However, in other embodiments, the operations 420, 422, and424 may be performed for a single commit cycle in the commit cycle queue217 or for commit cycles that include a predicted number of updatesabove a size threshold, e.g., above the second size threshold. Invarious embodiments, one or more of the operations 420, 422, and 424 maybe combined, omitted, or supplemented by an additional operation in aprocess for determining a progress point for a commit cycle.

In operation 420, it is determined whether a commit cycle that isestimated to take a relatively long time will start soon. A commit cyclemay be estimated to take a relatively long time because, based on itssignature, it is estimated to include a large number of updates. A sizethreshold number of updates may be used to estimate whether a commitcycle will take a relatively long time. In addition, available resourcesand the number of other jobs competing for those resources may beconsidered in operation 420. If there is a commit cycle that isestimated to take a relatively long period of time that has not yetstarted, the process 400 may advance to operation 419 where statisticsare immediately collected before this commit cycle begins. In someembodiments, statistics collection may not be scheduled to beginimmediately, but may be scheduled to begin in the near future.Accordingly, if a commit cycle is predicted to have a large number ofupdates and it is determined in operation 420 to have not yet started, astatistics collection time can be moved forward to an immediate time,even if the statistics collection time is scheduled for a later time.Otherwise, the process 400 moves to operation 422.

In operation 422, it is determined: (a) whether there is commit cyclethat is currently executing; (b) whether this currently executing commitcycle is predicted to include a large number of updates; and (c) whetherthis commit cycle is predicted to be near completion. For example, thesignature of a currently executing commit cycle indicates it averages50,000 updates. Database logs indicate that the currently executingcommit cycle has 46,000 pending updates. In this example, it may bepredicted that the currently executing commit cycle is near completion.The prediction may be made based on whether a percent complete isgreater than an upper threshold, e.g., 90 percent. For this example, theprocess 400 advances to operation 426. In operation 426, statisticscollection is delayed until the currently executing commit cyclefinishes and either commits or rolls back its pending updates.Otherwise, the process 400 moves to operation 424.

In operation 424, it may be determined whether there is a currentlyexecuting commit cycle that is in the “middle” of its life cycle. Inother words, the currently executing commit cycle is determined to nothave just started and also to not be near the end of its life cycle. Apercent complete may be compared to a lower threshold to determinewhether a commit cycle has just started. For example, if a percentcomplete is greater than a lower threshold, e.g., 10 percent, it may bedetermined that the currently executing commit cycle has not juststarted. Similarly, a percent complete may be compared to the upperthreshold (referred to above) to determine whether a commit cycle isnear completion. For example, if a percent complete is less than theupper threshold, e.g., 90 percent, it may be determined that thecurrently executing commit cycle is not near completion. If a currentlyexecuting commit cycle is determined to not have just started and alsoto not be near the end of its life cycle, then it may be inferred thatthe commit cycle is in the middle of its life cycle, and the process 400advances to operation 428. On the other hand, if it is determined thatthe currently executing commit cycle is not in the middle of its lifecycle, the process 400 advances to operation 419 where statistics may beimmediately collected.

In operation 428, it is determined whether the age of a current set ofstatistics is greater than a staleness threshold. If the age of thecurrent set of statistics is greater than the staleness threshold, thenin operation 432, statistics may be immediately collected usingsampling. Staleness refers to how many changes have been made since thelast time statistics were collected. In some embodiments, a measure ofstaleness and the staleness threshold may be represented as apercentage. Collecting statistics can require scanning and sorting allof the data in a table. Collecting statistics via sampling can use fewerresources and be faster because it does not require reading all of thedata in a table. Once the currently executing commit cycle that is inthe middle of its life cycle finishes, statistics can be collected inthe typical manner reading all of the data in the table. The statisticscollected based on sampling are only used for a short time until thecurrently executing commit cycle finishes. If the age of the currentstatistics is less than or equal to the staleness threshold, then inoperation 430, statistics collection may be delayed until the currentlyexecuting commit cycle finishes. In other words, the current statisticsmay be stale but may be deemed not so stale that updating the statisticscan wait until the currently executing commit cycle finishes.

Referring again to FIG. 2, the database management system 211 includes astatistics collection component (statistics collector) 215. Logic in thestatistics collection component may be configured to perform processes300 and 400. This logic may take the form of software components,hardware components, or both software components and hardwarecomponents.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

While the foregoing is directed to exemplary embodiments, other andfurther embodiments of the disclosure may be devised without departingfrom the basic scope thereof, and the scope thereof is determined by theclaims that follow.

What is claimed is:
 1. A computing device for determining a statisticscollection time for a database table, comprising: a memory to store adatabase and a database management system, the database managementsystem having a statistics collection component, the statisticscollection component including a data structure to store data associatedwith one or more commit cycles; a processor to cause the statisticscollection component to perform operations comprising: estimating a sumof predicted updates included in one or more commit cycles, wherein eachof the commit cycles is associated with a predicted number of updates;determining whether the estimated sum of predicted updates is greaterthan a first threshold; determining a progress point for a first one ofthe commit cycles; and selecting a time to collect statistics based onthe progress point of the first commit cycle.
 2. The computing device ofclaim 2, the processor further causing the statistics collectioncomponent to determine: whether the first commit cycle includes apredicted number of updates that is greater than a second threshold,wherein the determining of a progress point for the first commit cycleis performed in response to determining that the first commit cycleincludes a predicted number of updates that is greater than the secondthreshold.
 3. The computing device of claim 2, wherein the determiningof a progress point for the first commit cycle includes: determiningwhether the first commit cycle has not yet begun, and selecting acurrent time as a time to collect statistics in response to determiningthat the first commit cycle has not yet begun.
 4. The computing deviceof claim 2, wherein the determining of a progress point for the firstcommit cycle includes: determining that the first commit cycle hascompleted more updates than a lower threshold of pending updates,determining that the first commit cycle has completed fewer updates thanan upper threshold of pending updates, in response to determining thatthe first commit cycle has completed more updates than a lower thresholdof pending updates and determining that the first commit cycle hascompleted fewer updates than an upper threshold of pending updates,determining whether an age of a current set of statistics is less thanor equal to an staleness threshold, and selecting a future time as atime to collect statistics in response to determining that the age ofthe current set of statistics is less than or equal to the stalenessthreshold.
 5. The computing device of claim 2, wherein the determiningof a progress point for the first commit cycle includes: determiningthat the first commit cycle has completed more updates than a lowerthreshold of pending updates, determining that the first commit cyclehas completed fewer updates than an upper threshold of pending updates,in response to determining that the first commit cycle has completedmore updates than a lower threshold of pending updates and determiningthat the first commit cycle has completed fewer updates than an upperthreshold of pending updates, determining whether an age of a currentset of statistics is greater than an staleness threshold; and selectinga current time as a time to collect statistics using sampling inresponse to in response to determining that the age of the current setof statistics is greater than the staleness threshold.
 6. The computingdevice of claim 2, wherein the determining of a progress point for thefirst commit cycle includes: determining that the first commit cycle hascompleted more than an upper threshold of pending updates, and selectinga future time as a time to collect statistics, wherein the future timeis a time when the first commit cycle has completed.
 7. The computingdevice of claim 2, wherein the estimating of a sum of predicted updatesincluded in one or more commit cycles includes: monitoring the one ormore commit cycles to determine whether a number of updates for a firstcommit cycle diverges from a predicted number updates for the firstcommit cycle, and revising a sum of predicted updates associated withthe one or more commit cycles in response to determining that the firstcommit cycle diverges from the predicted number of updates for the firstcommit cycle.
 8. The computing device of claim 2, wherein thedetermining a progress point for a first one of the commit cyclesfurther comprises: determining a respective progress point for each aplurality of the one or more commit cycles, and sorting the plurality ofcommit cycles into a first order according to a predicted number ofupdates for each commit cycle, wherein the determining of a respectiveprogress point for each of the plurality of commit cycles are madeaccording to the first order.
 9. A computer program product fordetermining a statistics collection time for a database table, thecomputer program product comprising a computer-readable storage mediumhaving program instructions embodied therewith, the program instructionsexecutable by a processor to cause the processor to: estimate a sum ofpredicted updates included in one or more commit cycles, wherein each ofthe commit cycles is associated with a predicted number of updates;determine whether the estimated sum of predicted updates is greater thana first threshold; determine a progress point for a first one of thecommit cycles; and select a time to collect statistics based on theprogress point of the first commit cycle.
 10. The computer programproduct of claim 9, wherein the determining of a progress point for thefirst commit cycle includes: determining whether the first commit cyclehas not yet begun, and selecting a current time as a time to collectstatistics in response to determining that the first commit cycle hasnot yet begun.
 11. The computer program product of claim 9, wherein thedetermining of a progress point for the first commit cycle includes:determining that the first commit cycle has completed more than an upperthreshold of pending updates, and selecting a future time as a time tocollect statistics, wherein the future time is a time when the firstcommit cycle has completed.
 12. The computer program product of claim 9,wherein the estimating of a sum of predicted updates included in one ormore commit cycles includes: monitoring the one or more commit cycles todetermine whether a number of updates for a first commit cycle divergesfrom a predicted number updates for the first commit cycle, and revisinga sum of predicted updates associated with the one or more commit cyclesin response to determining that the first commit cycle diverges from thepredicted number of updates for the first commit cycle.